A Real-time Visual Mosaicking and Navigation System

A real-time visual mosaicking and navigation system for use near the sea floor has been developed and deployed as a pilot aid on MBARI ROVs. This system provides high-precision, environment-relative vehicle positioning and control without the use of external positioning arrays. The system uses a live video feed from a camera on the ROV combined with velocity data from an acoustic Doppler velocity log (DVL) to build and display in real time a mosaic depicting the sea floor beneath the vehicle. The mosaic is composed of individual video frames (tiles) snapped at appropriate intervals and placed in a larger composite image. The tile location is determined by image disparities from frame-to-frame image correlation combined with DVL navigation data. The result is a visual map showing the current vehicle position, which is robust to visual outages, such as dust clouds or periods out of visual range of the seafloor. This map can be used by the ROV pilot to navigate the seafloor environment, either via direct joystick control of the vehicle or through an automatic control interface allowing a point of interest in the mosaic to be selected and autonomously moved to. To date, this system has been used as a pilot aid for ROV operation. With some extensions, it has the potential to be deployed on AUVs as well. This paper reviews the high-speed vision algorithms used to calculate in real time camera displacement in the challenging visual environment at the sea floor (documented in part in previous papers) and the use of these algorithms in an automatic stationkeeping and online mosaicking system for visual odometry. The paper then focuses on new results from continued development of the system. Issues encountered during field testing are presented, along with the methods developed to address these issues. In particular, the incorporation of DVL measurements is described. These measurements provide a means to compute vehicle position during vision outages and allow the online mosaic to be continued when vision is restored. Using this complementary sensor has not only increased the robustness of the system to outages in the primary sensor, but has also improved general vehicle control by providing direct velocity feedback to the vision sensor. The combined system provides direct measurements with respect to the vehicle environment along with an intuitive and information-laden display for human users.